Fishing for Exactness

Statistical methods for automatically identifying de pendent word pairs i e dependent bigrams in a cor pus of natural language text have traditionally been performed using asymptotic tests of signi cance This paper suggests that Fisher s exact test is a more ap propriate test due to the skewed and sparse data sam ples typical of this problem Both theoretical and experimental comparisons between Fisher s exact test and a variety of asymptotic tests the t test Pearson s chi square test and Likelihood ratio chi square test are presented These comparisons show that Fisher s exact test is more reliable in identifying dependent word pairs The usefulness of Fisher s exact test ex tends to other problems in statistical natural language processing as skewed and sparse data appears to be the rule in natural language The experiment presented in this paper was performed using PROC FREQ of the SAS System